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^SJ , Abstract 

We show how a simple scheme of symbohc dynamics distinguishes 
a chaotic from a random time series and how it can be used to detect 
structural relationships in coupled dynamics. This is relevant for the 
question at which scale in complex dynamics regularities and patterns 
P^ ■ emerge. 



1 Symbolic dynamics distinguishing chaotic from 
random dynamics 

Q.^ ■ 1.1 Random sequences and dynamical iterates 



According to many popular accounts, chaotic dynamics seem to blur the dis- 
tinction between determinism and randomness. While following a fixed rule, it 
is characteristic of chaotic dynamics that in the longer term no prediction of the 
iterates of given initial values is possible, and it therefore seems that sequences 

f~^ [ of points generated by chaotic dynamics are difficult, if not impossible, to dis- 

tinguish from random sequences. Of course, this is not so, and one may exploit 
regularities in the relationships between subsequent points in the sequence to 

j^ ' extract useful information about the underlying dynamics. By now, very so- 

H I phisticated methods have been successfully developed, and we refer to [10] for 

a good account of the state of the art, describing both the older linear and the 
more recent non-linear tools, in particular phase space and other embedding 
methods, together with a rich spectrum of applications. 

It is the purpose of the present article to analyze the relationship between ran- 
domness and chaos in an elementary manner using simple symbolic dynamics, 
and to utilize this to elucidate the formation of higher level structures through 
the coordination of lower level non- linear dynamics, as initiated in our earlier 
contribution |2|. 



*aIso; Santa Fe Institute, USA; jost@niis.mpg.de 




Figure 1: The graph of the tent map f{x) has its peak at a; = 1/2 and intersects the 
diagonal ai x — 2/3. 



The baseline situation is a sequence Xn, with n € N as usual, of points ran- 
domly drawn from the unit interval [0, 1], independently of each other and all 
distributed according to the uniform density. The latter means that for each 
subinterval of [0, 1], the probability of finding Xn, for given n, in that interval is 
equal to its length. 
We then consider the tent map 



/ 



[0, 1] ^ [0, 1] 

J2x for < a; < 1/2 
[2-2j; for 1/2 < a; < 1 



as the basic example of a chaotic iteration 



x{n + 1) = f{x{n)) for n G N. 



(1) 



(2) 



(In this paper, whenever we discuss a concrete map, / will always be the tent 
map.) Its stationary density p on [0, 1] is the uniform density 



p{x) = 1. (3) 

This means that if, for some generic initial value x(0)|j we randomly choose 



n £ N and the corresponding point from the sequence x(n), then again for each 

;erval of [0, IJ, 
equal to its length|3 



subinterval of [0, Ij, the probability that the point lies in that subinterval is 



^This qualification is needed because for all initial values of the form x'(0) = (1/2)^" for 
some V G NU{0}, the iteration will end up in the fixed point 0. Those particular initial values, 
however, constitute a set of measure in [0, 1] and can therefore be neglected for the purposes 
of our discussion. 

■^It might seem that there is a profound difference between the ways the points are selected 
in the random and in the dynamical case. In the former one, we choose a random point for 



1.2 Symbolic dynamics derived from time series 

We use the stationary density p to construct derived symbolic dynamics accord- 
ing to the foUowing rule, for some a G (0, 1), 

.(.) = (» '!»f J" (4) 

II II a < x < 1. 

So, from a sequence Xn in [0, 1], one obtains a derived symbolic dynamics s„ = 
s(xn) G {0, !}• That sequence x„ can now either be a random sequence chosen 
according to the density p, that is as above in our baseline situation, or a 
sequence x(n) coming from our chaotic iteration ^. 

The most natural choice for the partition point a seems to be 1/2. In that case, 
however, the symbolic dynamics does not distinguish between the random and 
the chaotic sequence. For our random sequence x{n), when, say, s„ = 0, then 
< x{n) < 1/2, and each of the two subcases < a; < 1/4 and 1/4 < a; < 1/2 
occurs with probability 1/2; in the first case, Sn+i = 0, while in the second 
one, Sn+i = 1, and so the two possible values for Sn+i both occur with equal 
probability 1/2. Since the same happens in case s„ = 1, this is independent of 
the value of s„, as for a random sequence. 

The situation changes for other partition points a|f| The most significant and 
easy case is a = 2/3 as the graph of the tent map intersects the diagonal there, 
see Figure 1; so, we consider 

For the random sequence, the values &„ — and s„ = 1 occur independently 
with probabilities 2/3 and 1/3. For the chaotic sequence, when s„ = 1, that is, 
2/3 < x{n) < 1, the rule ^ for the tent map yields < x{n + 1) < 2/3, that 
is Sn+i = 0. Thus, the successor of state one 1 is always state for the chaotic 
map; no transition from 1 to 1 is possible. When we have the state s„ — 0, both 
transitions are equally likely: when < x{n) < 1/3, we have < x{n + l) < 2/3, 
that is, Sn+i = 0, while for 1/3 < a: < 2/3, we get s„+i = 1. Thus, the state 
transition probabilities satisfy 

p(0|0) = p(l|0) = 1/2, p(0|l) = 1, p{l\l) = (6) 

for the symbolic dynamics derived from the chaotic one while for the random 
one the probabilities p{0) = 2/3 and p{l) = 1/3 are independent of the previous 
state, that is, 

p(0|0) = p(0|l) = 2/3, Pirn - p{l\l) = 1/3. (7) 



fixed "time" n, while in the latter one, we choose a point fixed by the dynamics at a random 
time n. The fundamental concept of ergodicity (which applies to our example), however, tells 
us that this leads to the same, that is temporal averaging is equal to spatial averaging. 

^In |lll . the difference between homogeneous partitions (based on the underlying Lebesgue 
measure) and generating partitions has been analyzed. In the present case, however, the 
partition point a = 1/2 yields both a homogeneous and a generating partition. 



More generally, for a partition point a, the state probabilities are p(0) = 
a,p{l) = 1 — a. For the random dynamics with a uniform density, we have the 
transition probabilities 

p(0|0) = p(0|l) = a, and p(l|0) = p{l\l) = (1 - a) (8) 

For the tent map, we have 

Pi0\0)=pim ^ i p(l|l) = ,1-^, P(0|1) = — ^ for a < 2/3 
2 2(1 — a) 2(1 — a) 

p(0|0) = ^^^, p(l|0) = i^, p(l|l) = 0, p{Q\l) = 1 for a > 2/3. (9) 
a a 

1.3 Symbolic dynamics from ordering relations between 
subsequent points 



The basic idea here has been first introduced by Bandt and Pompe |4Ji and is 
readily described by taking two points a;^,x^ G [0, 1] and the symbolic rule 

II if x^ < x\ 

We apply this to our random sequence, that is, at each step, we take x^ = 
Xn-jX^ — Xn+\- Thus, wc draw the points x^^x^ randomly and independently. 
The state probabilities are again p(0) = p{\) — 1/2, but the transition proba- 
bilities become different: 

p(0|0)=p(l|l) - 1-^ 

p(l|0)=p(0|l) - -^ (10) 

because when x\ is random, the average value of those X2 with xi < X2 is ^. 
In fact, the symbolic dynamics is not Markovian since, for example p(0|00) < 
p(0|01). More generally, the more Os have already occurred in sequel, the less 
likely it gets to observe another as the next state. Thus, state probabilities 
depend on the entire past of the sequence. In particular, the symbolic sequence 
derived from our random sequence is not random itself. 

The situation becomes simpler when we derive the symbolic dynamics from 
our chaotic map, x^ — x(n\x^ = x{n + 1). Here, the probabilities p(0) and 
p(l) are again equal, and the transition probabilities are given by ([6]), because 
X < f{x) precisely if < a; < 2/3. The process now is Markovian. For example, 
p(0|00) = p(00|0) = p(OOO) = p(lOO) = p(00|l) = p(0|10)l!l three consecutive 

*even though they used it for a somewhat different purpose, as a method for approximating 
the entropy of a time series, instead of for a distinction between random and chaotic sequence 
as we shall do here 

^We write the symbolic sequences here from left to right, that is, 10 means that we first 
see the symbol 1 and then the symbol 0; this explains the reversals of symbol order in these 
equations because conditioning is written from right to left. 



points a;(n),a;(n+l),a:(n + 2) are between and 2/3 precisely when < x{n) < 
1/6 while the symbolic sequence 100 occurs when 5/6 < x(n) < 1. 
The pattern becomes even more obvious when we consider three consecutive 
points x^,x^,x^ and the symbol dynamics defined by 

s{x'^,x'^,x^) = 1 a x'^ < x"^ < x^ 

s{x'^ , x'^ , x^) = 2 ifx^<a;^>a;^ 

s{x'^,x'^,x^) = 3 ifx^>a;^>a;^ 

s{x\x^,x^) = 4 [ix^>x^<x^. (11) 

(For simplicity, we neglect all cases of equality from now on because those occur 
with probability 0.) 

Regardless of how the points x^ ,x'^ , x^ are drawn, the only possible transitions 
are 11, 12, 23, 24, 33, 34, 41, 42. When the points are randomly drawn, all 
of them occur. The transition probabilities are different, however: for example 
p(l|l) < p(2|l). As before, the process is not Markovian in that case: for 
example p(l|ll) < p(l|14). 

When the points are obtained from the chaotic iteration, x^ — x{n), x"^ = x{n-\- 
1), x^ = x{n+2), state 3 can no longer occur because we have already seen above 
that when x{n+ 1) < x{n) we have 2/3 < x{n) < 1 and < x{n+ 1) < 1/3 and 
therefore x{n+2) > x(n+l). (In fact, the states 1 and 2 in the present dynamics 
correspond to the state for the above symbolic dynamics obtained from the 
ordering between two consecutive points from a chaotic dynamics, while state 
4 corresponds to state 1 in that latter dynamics.) Thus, this derived symbolic 
dynamics leads to an easy distinction between the random and the chaotic ones. 

1.4 Generalities 

The preceding makes possible a distinction between a particular chaotic itera- 
tion, the tent map, and a random iteration with the same underlying probability 
density. The question arises whether this symbolic method can also distinguish 
a more general class of chaotic iterations from a random, that is, to what extent 
this is useful for distinguishing chaos from randomness. One generalization is 
clear: the symbolic dynamics derived from ordering relations between consecu- 
tive points applies to any chaotic map that is conjugate to the tent map, like 
the logistic map. Of course, the stationary density will no longer be uniform 
in general, but it can readily be estimated from the time series produced by 
the dynamics, and one can take the random iteration based on that probability 
density for comparison. 

For the symbolic dynamics derived from the partition, one should know the op- 
timal partition point a; of course, when a is unknown, one can try different ones 
so as to minimize the entropy of the resulting symbolic transition dynamics. For 
the random dynamics with a uniform density and partition point a, we had the 
state probabilities p(0) = 2/3, p(l) = 1/3 and independent transitions p{i\j)', 
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Figure 2: Transition entropies {Hd and Hr) for the tent map and the corresponding 
random system, calculated from Eq. (|12|l . by using ((8]) and ([9} respectively. 



this gives the entropy 



H=-J2 P^) J2 PiM^ogpiJli) = log 3 - 2/3 - 

1=0,1 j=0,l 



.918. 



(12) 



For the tent iteration, we had p(0|l) = l,p(l|l) = 0,p(l|0) = p(0|0) = 1/2, 
leading to the entropy 

H = 2/3 (13) 

which is significantly smaller. In general, we should expect that the symbolic 
dynamics for a chaotic map leads to a smaller entropy than for a random one 
(based on the same probability density). In fact, for the present example, the 
entropy difference between the random and the chaotic sequence is largest for 
a = 2/3, see Figure 2. In contrast, for a — 1/2 (which corresponds to the 
generating partition for the tent dynamics) , the entropies are the same and the 
difference vanishes. When a is close to or 1, the entropies of the random and 
of the chaotic sequence both become quite small, and therefore, the difference 
is likewise small. Our strategy 5A is to choose such an a that the difference 
is maximal so as to make the difference between random and chaotic dynamics 
most pronounced. 

One can also view this in the following manner. When the baseline is some 
constant or similarly trivial dynamics, then the entropy difference between the 
chaotic tent nrap and the trivial map is largest for the value of a that corresponds 
to the generating partition, that is, for a ~ 1/2; in fact, one may consider 
this as the definition of the generating partition. This, however, then cannot 



distinguish between a deterministic chaotic iteration and a random sequence. 
When we want to find such a distinction, we should look rather for a partition 
where the entropy difference between those two sequences is maximized, and 
that lead us to the value a = 2/3 in the present case. This is a very simple 
instance of the principle that interesting structure is neither trivial nor random. 
While for more general chaotic dynamics, in general it is not easy, or perhaps 
even not possible, to find the optimal partition, still any partition that leads to 
an entropy difference between a chaotic iteration and a random sequence with 
the same underlying density yields detectable symbolic differences. Therefore, 
our method possesses some generality, and as an example, we have applied it to 
the Henon map in our recent work [7]. 

2 Coupled dynamics 

In order to move beyond the simple comparison between a random and a chaotic 
sequence, we now consider coupled maps, as in [2]. This means that we take 
some graph F, unweighted and undirected for simplicity, with N vertices or 
nodes. Vertices x,y oiT that are connected by an edge of F are called neighbors, 
symbolically denoted hy x ^ y. The number of neighbors of x is denoted by Ux- 
For a parameter e, the coupling leads to the system 

x{n + 1) = fixin)) + — ^(/(2/(n)) - /(x(n))). (14) 

nx — 

y^x 

Thus, X now adjusts its state not only the basis of its own present state, but 
also takes the state differences from its neighbors into account. The coefficients 
on the right hand side are chosen in such a manner that the total weight of all 
the contributions is 1, that is, the same as in ^. 
Rewriting P^ as 



x{n + 1) = (1 - e)f{x{n)) + — J^ f{y{n)) (15) 



y^x 



leads to an alternative interpretation. Here, the node x updates its state on the 
basis of a weighted average of a function of its own state and the corresponding 
values from its neighbors. In the special case where the graph F is complete, 
that is, each of the N vertices is connected with all A^ — 1 other ones, when we 
then choose e = ^^^^ we obtain 

zev 

Thus, in this particular case, the r.h.s. of the iteration dynamics equation is 
the same for all the vertices. Since then each of them updates its state not 
only by the same rule, but also with the same input, their states are all equal, 
that is x{n + 1) = y{n + 1) for any two vertices x,y. Thus, the network is 



synchronized. It then turns out that synchronization also occurs for other values 

of the coupling strength e, see [9], or for other graph topologies and is stable 

against perturbations, see e.g. [5|. 

So, the conceptually simplest possibilities for the resulting network dynamics 

are: 

1. The individual dynamics x{n) are completely unrelated. This happens 
for e = 0. In that case, each node behaves chaotically and is completely 
independent of the other ones. 

2. The nodes synchronize, that is, x{n) = y{n) for all nodes x, y G F. In that 
case, the sum on the right hand side of p^ becomes 0, and consequently, 
each node behaves according to 

x{n + 1) = f{x{n)) (17) 

which is the same as in the uncoupled case. 

Thus, in both the uncoupled and the synchronized case, the individual dynamics 
are the chaotic ones given by ^. From looking at an individual node, we are 
not able to distinguish between the two scenarios. Both cases are extreme ones, 
and ultimately dynamically not very interesting, even though the synchroniza- 
tion of chaos after all is a surprising phenomenon. We therefore ask whether one 
can find and describe more interesting dynamics between those two extrema. 
At some level, such states should exhibit a behavior intermediate between the 
uncoupled and the fully synchronized dynamics. In f2], we described some emer- 
gent behavior on a longer time scale when transmission delays were introduced 
in (|14p . Here, we shall look for behavior that is intermediate regarding either 
the spatial coordination or the one of the state values x{n). The paradigm for 
partial spatial coordination is the formation of dynamical clusters such that the 
nodes inside a cluster synchronize or otherwise coordinate their states, but that 
no such coordination occurs between clusters. An example of partial state value 
synchronization is the phase synchronization detected in 5^ where the dynami- 
cal states x{n) have their individual local temporal minima (or maxima) at the 
same times. 

We shall now describe how those two types of dynamic behavior correspond to, 
and therefore can be detected by, certain types of derived symbolic dynamics 
according to our above scheme. We ask two questions: 

1. In which settings or constellations do the symbolic dynamics derived from 
the state dynamics of some vertex exhibit regularities or characteristic 
features distinct from both the random ([7]) and the isolated chaotic one 

dlD? 

2. Under which circumstances, beyond the obvious one of synchronization, 
do the symbolic dynamics at different vertices show some correlations? 



2.1 Local symbolic dynamics detecting collective proper- 
ties of the dynamical system 

We describe here three different types of relationships between local symbolic 
dynamics - evaluated at a single node - and collective properties of the dynam- 
ical system 

1. Local symbolics and complexity of the collective dynamics: The 
largest Lyapunov exponent 

The Lyapunov exponents measure the rates of stretching or shrinking 
in a possibly high dimensional dynamical system. A positive Lyapunov 
exponent indicates an expanding, a negative one a contracting direction. 
A positive Lyapunov exponent is considered as an indication of chaos, and 
when there is more than one positive Lyapunov exponent, one speaks of 
hyperchaos. Lyapunov exponents are often difficult to compute in practice. 
We have found that the transition probability for the symbolic rule ([5]) at 
any node of the dynamical network qualitatively matches the behavior of 
the largest Lyapunov exponent as becomes evident in Figure 3|j 

For the uncoupled tent map, we hadp(0|0) = 1/2, see ([6]), and it is remark- 
able that when the largest Lyapunov exponent decreases, this transition 
probability can even become 0, that is, two successive Os no longer occur. 
The important point here is that some very easy measurement at one sin- 
gle node yields qualitative information about a global characteristic of the 
network that itself is difficult to compute. 

2. Local symbolics and coordination of individual dynamics in the 
network: Phase synchronization 

We say that two nodes i,j are phase synchronized when the temporal 
maxima of x^{n) and x^ (n) occur for the same values of n, that is, simul- 
taneously; and we may require the same for the minimal This is most 
easily detected by the symbolic dynamics dill) because phase synchroniza- 
tion means that the symbols 2 and 4 for the corresponding symbolics oc- 
cur simultaneously, and therefore also the other symbols by the transition 
constraints for (jlip . Phase synchronization is weaker than full synchro- 
nization, and therefore can occur more easily, that is for a wider range of 
coupling strengths and networks. It is a property of the state dynamics at 
a coarse level that may not be evident when focusing of the precise values 
of the states, that is, at the fine scale. Thus, the important point here 
is that the symbolics easily reveal a qualitative property at some coarse 
scale of the state dynamics. 



^In [4], a qualitative similarity between the permutation entropy obtained from symbolic 
rules of the type (IIOI I , JTTJ and the Lyapunov exponent of a time series derived from a single 
chaotic oscillator had been observed. 

^There exist different notions of phase synchronization in the literature, appropriate under 
different circumstances, see e.g. [l]. For our purposes, the one adopted here is most useful. 




Figure 3: The largest Lyapunov exponent (•) as a global measure of coupled dynam- 
ics and the transition probabilities p(0|0)(o) of local symbolic dynamics, for various 
networks. The horizontal line represents coupling strengths. Figures are plotted for 
(a) a globally coupled network with A'' = 50, (b) a scalefree network with A*' — 200 
and average degree 20, (c) and (d) for small world networks with TV = 200 and average 
degree 10 and 40 respectively. 



3. Local symbolics and regularities on larger temporal and spatial 
scales: 

As explained in [2], the coupling, possibly in conjunction with transmis- 
sion delays, may produce regularities at a longer time scale than accessible 
to the uncoupled individual dynamics that by their chaotic nature blur all 
distinctions on longer temporal scales. This must translate into a longer 
memory span of the symbolics. Conversely, memory effects, that is, long 
time correlations in the symbolics indicate a relevant longer temporal scale 
for the coupled dynamics. 

Concerning larger spatial scales, it should be worth investigating the sym- 
bolic dynamics in hierarchical structures as investigated e.g. in |12j . 
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2.2 Homogeneities at the symbolic level 

The issue of phase synchronization just described can also be considered in the 
hght of the second question raised above, namely the one about correlations 
in the symbolics. Obviously, when the network dynamics is synchronized, then 
so are the symbolics. But even when we do not have full synchronization, we 
should expect that the coupling leads to some coordination between the dynam- 
ics of the various nodes, and that should be detectable by suitable correlation 
measures. It is then natural to look at correlations between the symbolics, as 
asked above. Phase synchronization means that the symbolics of the different 
nodes become identical, but also the existence of dynamical regimes with weaker 
correlations between the symbolics is conceivable. It turns out that, remark- 
ably, the transition probabilities for the symbolics at a single node can again 
give some indication of the degree of homogeneity of the symbolics across the 
network. That match, however, is not perfect; it works only for a certain range 
of values for the coupling strength e for a given network. 

2.3 Symbolic dynamics as derived dynamics at a higher 
level of abstraction 

Let us contemplate the general problem: The symbolic dynamics is derived 
from a lower level state dynamics and thus not autonomous. For the issue 
of emergence, it would be desirable that this dynamics at a higher level of 
abstraction develops at least some degree of autonomy, that is, that subsequent 
symbol values, or at least their probabilities, can be predicted from the values 
at previous times. For the probabilities, this is possible in the isolated case, see 
^. The question remains whether this also emerges at the collective level. 
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